dasLLVM: bool and pointer reinterprets, default<T[N]>, try/recover return, inline ptr_diff, speculatable lookups, --jit-remarks by borisbat · Pull Request #4472 · GaijinEntertainment/daScript

borisbat · 2026-11-07T06:05:14Z

Behavior change: a return inside a try or recover arm is a compile error on every tier (error 30299) - set a variable in the arm and return after the pair. A closure's own return inside the arm stays allowed.

Why. Seven of lookibed's JIT reports: reinterpret<uint8>(true) read 0xff and failed the verifier when it fed a call; a return inside a try arm left the arm and the function answered wrong; a = default<int[4]> zeroed one element; reinterpret<uint>(p) emitted trunc ptr; p - q was a slot call with a run-time stride (0.32 s against 0.0045 s); a global read under a branch in a loop kept its lookup in the loop (0.405 s against 0.042 s); LLVM's optimization remarks had no switch.

What changes.

A bool-to-integer reinterpret zero-extends and integer-to-bool compares the low byte with zero. The test asserts a bool byte of 2 only under -jit: the interpreter and AOT read such a byte as a C++ bool, which a Debug build reads as its low bit.
The front end rejects a return inside a try/recover arm; the tests that pinned the interpreter's old return-from-try behavior keep their break and continue cases. AstSerializer::getVersion() moves to 246, so a cached module holding the shape is not served past the new check.
default<T[N]> zeroes the declared shape when no element is provided.
A pointer reinterprets to any integer width through ptrtoint.
i_das_ptr_diff is an intrinsic row: ptrtoint, sub, division by the constant stride.
jit_get_global_mnh and jit_get_shared_mnh are speculatable, so LLVM hoists a lookup out of a loop.
--jit-remarks=, --jit-remarks-missed= and --jit-remarks-analysis= print LLVM's remarks and run codegen past both caches.

Observable behavior.

print("{reinterpret<uint8>(true)}") under -jit: 0xff -> 0x1.
try { return 1 } recover { }; return 10: the interpreter returned 1 and the JIT 10 with a warning -> compile error on both.
a = default<int[4]> on [1,2,3,4] under -jit: [0,2,3,4] -> [0,0,0,0].
reinterpret<uint>(p) under -jit: Invalid bitcast -> the low 32 bits.
The pointer-difference loop: 0.32 s -> 0.0045 s; the branch-in-loop global read: 0.405 s -> 0.042 s.
daslang -jit x.das -- --jit-remarks-missed=loop-vectorize prints loop not vectorized: ... per loop.

Where to look. visitExprCast, preVisitExprMakeStruct and preVisitExprReturn in modules/dasLLVM/daslib/llvm_jit.das; the i_das_ptr_diff row in llvm_jit_intrin.das; the extern attributes in llvm_jit_common.das.

Validation, claims, ledger

Validation

The module-owned suite at the tip: bin/daslang dastest/dastest.das -- --test modules/dasLLVM/tests 170 passed, 5 skipped; tests/jit_tests under -jit 435 passed, 1 skipped.
Full AOT sweep: preflight --only tests-aot (the test_aot build and sweep over tests/) passed at the tip, 551 s; per-PR CI builds only the subset.
The three fixed emitter paths were negative-controlled: each test fails with its branch mutated and passes restored.
Codex review round at the emitter-only tip: no findings, every changed file in its trace. The front-end change came after it: tests/ interpreted 15588 passed, 9 skipped; tests/language under -jit 1981 passed.
The tree-sitter parser lane runs on ubuntu-24.04: on ubuntu-latest (26.04) the swift toolchain the test action installs cannot load libxml2.so.2, and the lane only runs when a PR touches tree-sitter-daslang/, which the example edit here does.
The sequence example's smoke clones dasCards main, which returned from inside a try arm; the restructure merged upstream, so the darwin15 lane goes green on a rerun.
Two docs sub-gates are red on this box for reasons outside the diff: docs/digest (master's shipped digest is stale, the regeneration adds the AstConstVisitor entries an earlier merge introduced) and docs/sphinx-html (the PATH sphinx-build lacks sphinx_sitemap).

Claims - stated, not tested

A remark without --jit-debug prints its location as <unknown>:0:0; the function name the issue asks for needs a custom diagnostic handler, which needs a C++ callback binding dasLLVM does not expose.

Not done

JIT: a loop over float? bounded and fed through int? reloads both ints after every float store and never vectorizes - the emitter gives LLVM no alias information for typed pointers (no !tbaa, no noalias on pointer parameters) #4446 typed-pointer alias information: the test lands skipped behind a flag. daslang keeps -fno-strict-aliasing semantics (a pointer is an address), so no type-based alias metadata; noalias where the language can prove it is a separate PR after this arc.
A lowering of return inside a try arm (a return slot threaded through the arm's captures) is not attempted; the error is the ruling.
The rename duty for a ManagedVectorAnnotation or ManagedStructureAnnotation name string belongs to the folder that creates the annotation (src/builtin/REVIEW.md and module binding folders); this PR narrows the AST-headers rule and adds no rule there.
llvm_jit_sanitize.das keeps its own dump_ir: it passes --jit-compile-only and accepts a non-zero exit, which the shared jit_dump does not.

#nightly - the emitter set is pinned and the JIT sweep lanes are nightly.

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🔵 Needs a closer look

Because the changes alter JIT codegen semantics and add LLVM attributes like speculatable whose safety depends on subtle LLVM rules and cross-target runtime invariants, the review evidence calls for domain-expert sanity checks and broader validation before proceeding.

No open issues 🎉

What changed in this PR

This pull request updates dasLLVM’s JIT to fix several correctness edge cases (reinterpret lowering, fixed-array defaults, and return behavior in try/recover arms), improves performance for pointer-difference and global/shared-global lookup patterns, and adds --jit-remarks* flags to surface LLVM optimization remarks. It also adds/updates regression tests and documentation around these behaviors.

Changes:

Correct JIT lowering for reinterpret (bool↔integer semantics; pointer→integer via ptrtoint) and for default<T[N]> so fixed-size arrays zero the full declared shape.
Make return inside try/recover arms a hard JIT compile error (while preserving valid closure returns inside arms).
Add inline lowering for pointer difference (i_das_ptr_diff), enable remark printing via --jit-remarks*, and adjust the JIT pipeline to run past caches when remarks are requested; expand/refactor dasLLVM module tests and fixtures accordingly.

Approval Readiness

This pull request requires changes.

Moderate: speculatable on jit_get_global_mnh/jit_get_shared_mnh must be safe for speculative execution (no traps). The review notes the call can dereference ctx and may assert on miss; please make the preconditions explicit (e.g., add nonnull on ctx at least) to satisfy the attribute’s contract.
Moderate: lookups_inside_loop test logic appears potentially fragile because it only exempts .preheader blocks; LLVM can emit other canonical preheader names (e.g. .lr.ph*), which can cause false failures.
Nits (comment hygiene): multiple top-level multi-line //! doc comments won’t attach to the following def/[test] def and should be converted to // (or moved into the function body / made single-line contracts where appropriate).

File	Summary
`tests/REVIEW.md`	Updates test/review guidance for spawned-child execution and cache/output isolation.
`tests/jit_tests/return_in_try_recover.das`	Adds regression coverage asserting JIT rejects `return` inside `try`/`recover` arms (and validates allowed closure-return behavior).
`tests/jit_tests/reinterpret_ptr_to_uint.das`	Adds regression coverage for pointer→integer reinterpret behavior under `-jit`.
`tests/jit_tests/default_fixed_array_assign.das`	Adds regression coverage ensuring `default<T[N]>` assignment zeros the entire fixed-size array.
`tests/jit_tests/bool_reinterpret_byte.das`	Adds regression coverage for bool↔integer reinterpret byte/zero-extension semantics under `-jit`.
`tests/jit_tests/_fixture_return_in_try_recover.das`	Adds a child-run fixture used to reproduce and validate the return-in-try/recover JIT compile error.
`modules/dasLLVM/tests/llvm_jit_remarks.das`	Adds tests validating `--jit-remarks*` output and the “run past caches” contract.
`modules/dasLLVM/tests/llvm_jit_ptr_diff_inline.das`	Adds tests asserting `i_das_ptr_diff` lowers to inline arithmetic (not an indirect/slot call).
`modules/dasLLVM/tests/llvm_jit_pointer_aliasing.das`	Adds currently-skipped scaffolding for future typed-pointer aliasing coverage.
`modules/dasLLVM/tests/llvm_jit_never_fast_math.das`	Refactors tests to use shared IR-dump/helpers infrastructure.
`modules/dasLLVM/tests/llvm_jit_global_lookup.das`	Extends/refactors tests validating global/shared-global lookup hoisting behavior.
`modules/dasLLVM/tests/llvm_jit_debug_info.das`	Refactors tests to shared helpers and adjusts debug/symbol expectations.
`modules/dasLLVM/tests/_remarks_root.das`	Adds a fixture program intended to emit LLVM optimization remarks.
`modules/dasLLVM/tests/_ptr_diff_root.das`	Adds a fixture program for pointer-difference lowering inspection.
`modules/dasLLVM/tests/_pointer_aliasing_root.das`	Adds a fixture program for typed-pointer aliasing scenarios (future/placeholder coverage).
`modules/dasLLVM/tests/_global_lookup_root.das`	Extends fixture to include loop/branch global and shared-global lookup patterns.
`modules/dasLLVM/tests/_exe_test_common.das`	Introduces/updates shared helpers for JIT execution, IR dumping, and IR parsing in tests.
`modules/dasLLVM/REVIEW.md`	Updates dasLLVM module review/checklist documentation.
`modules/dasLLVM/daslib/llvm_jit.das`	Implements core JIT lowering fixes (reinterpret behavior, fixed-array default zeroing, and return-in-try/recover refusal).
`modules/dasLLVM/daslib/llvm_jit_run.das`	Adjusts runtime behavior to ensure remark-enabled runs force recompilation (bypass caches).
`modules/dasLLVM/daslib/llvm_jit_plan.das`	Tracks remarks enablement and forces codegen to run past caches when requested.
`modules/dasLLVM/daslib/llvm_jit_link.das`	Routes remark-enabled runs through paths that bypass cached artifacts as intended.
`modules/dasLLVM/daslib/llvm_jit_intrin.das`	Adds intrinsic mapping/implementation for inline `i_das_ptr_diff`.
`modules/dasLLVM/daslib/llvm_jit_common.das`	Wires LLVM pass-remarks options and applies extern attributes (including `speculatable`) to selected calls.
`modules/dasLLVM/daslib/llvm_jit_cli.das`	Adds CLI flags: `--jit-remarks=`, `--jit-remarks-missed=`, `--jit-remarks-analysis=`.
`modules/dasLLVM/CMakeLists.txt`	Registers newly added/updated dasLLVM tests in the module build configuration.
`modules/dasLLVM/ARCHITECTURE.md`	Documents the remarks flags and updated JIT behavior/intent (including lookup hoisting behavior).

_{Review effort: Balanced}

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🔵 Needs a closer look

This change alters correctness/performance-sensitive LLVM JIT emitter IR/attribute semantics and related CLI/cache behavior in ways that require cross-target human validation beyond the coverage of the added tests and static diff review.

No open issues 🎉

_{Review effort: Balanced}

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟢 Approval recommended

All touched files were reviewed with only minor documentation-style nits noted and no unresolved functional concerns or blocking issues raised in the review evidence.

No open issues 🎉

_{Review effort: Balanced}

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟡 Changes recommended

A moderate active issue remains because the implementation enforces the return-in-try/recover compile error across all compilation modes via the type inference tryDepth > 0 check, which conflicts with the PR description’s claim that it is a -jit-only behavior change and needs either description alignment or mode-specific gating.

1 open issue

Description says JIT-only, but the check runs in all tiers · New

_{Review effort: Balanced}

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟡 Changes recommended

A documentation nit remains unresolved in ast_serializer.h where the serializer version comment mischaracterizes a behavior change (it’s now a hard compile error for return inside try/recover), which could confuse future cache-version audits.

2 open issues

_{Review effort: Balanced}

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🔵 Needs a closer look

A remaining issue was identified in include/daScript/ast/ast_serializer.h where a version-note comment is misleading about the behavioral change, indicating the documentation needs correction before the change can be considered ready.

2 open issues

_{Review effort: Balanced}

The cast emitted `bitcast i1 to i8`, which reads true as 0xff and fails the verifier when it feeds a call. A bool source is zero-extended to the integer width; an integer source compares its low byte with zero. Fixes #4471 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🔵 Needs a closer look

The changes span multiple cross-cutting compiler areas (type inference/control-flow semantics, AST cache/serialization versioning, and LLVM JIT codegen/CLI behavior), so correctness and performance require maintainer review and running the full multi-target test/JIT matrix beyond what static diff inspection can verify.

No open issues 🎉

2 resolved since last review

_{Review effort: Balanced}

The JIT compiles the two arms as block functions called through jit_try_recover, so a return in an arm left the arm and the function answered wrong, while the interpreter returned from the function. There is no lowering for the shape without a return slot threaded through the arm's captures, so the front end now rejects it on every tier (error 30299, naming the fix: set a variable in the arm and return after the pair). A closure's own return inside the arm is the closure's and stays allowed. The flow test keeps its break and continue cases; the return cases are gone with the semantics. Fixes #4462 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

reinterpret<uint>(p) built `trunc ptr to i32`, which the verifier refuses; only the 64-bit target went through ptrtoint. Every integer target does now, and ptrtoint to a narrower width keeps the low bits, as the interpreter does. The JIT half of #4108; the interpreter half is the reinterpret eval-slot fix on the optimizer branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The make-struct arm zeroed max(elements, 1) strides; for default<T[N]> there are no elements and a stride is one element, so only the first element was cleared and `a = default<int[4]>` left the rest. The arm now zeroes the declared shape's size when no element is provided and reads the element type through the fixed-array chain for the empty-tuple case, as the interpreter does. Found by the regex rewrite: a set reset through default<> under -jit kept its old bits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

ubuntu-latest moved to 26.04, where the swift toolchain the parser test action installs fails to load libxml2.so.2 and swift-build exits 127. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🔵 Needs a closer look

The change affects high-impact core compiler semantics and LLVM JIT/codegen behaviors (including marking runtime helper globals as speculatable), which requires maintainer verification across supported platforms/configurations beyond what static diff review can safely confirm.

No open issues 🎉

_{Review effort: Balanced}

…turn, inline ptr_diff, speculatable lookups, --jit-remarks (GaijinEntertainment/daScript#4472) * jit: a reinterpret between bool and integer is the byte, not a bitcast The cast emitted `bitcast i1 to i8`, which reads true as 0xff and fails the verifier when it feeds a call. A bool source is zero-extended to the integer width; an integer source compares its low byte with zero. Fixes GaijinEntertainment/daScript#4471 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * infer: a return inside a try or recover arm is a compile error The JIT compiles the two arms as block functions called through jit_try_recover, so a return in an arm left the arm and the function answered wrong, while the interpreter returned from the function. There is no lowering for the shape without a return slot threaded through the arm's captures, so the front end now rejects it on every tier (error 30299, naming the fix: set a variable in the arm and return after the pair). A closure's own return inside the arm is the closure's and stays allowed. The flow test keeps its break and continue cases; the return cases are gone with the semantics. Fixes GaijinEntertainment/daScript#4462 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: a pointer reinterprets to any integer width through ptrtoint reinterpret<uint>(p) built `trunc ptr to i32`, which the verifier refuses; only the 64-bit target went through ptrtoint. Every integer target does now, and ptrtoint to a narrower width keeps the low bits, as the interpreter does. The JIT half of GaijinEntertainment/daScript#4108; the interpreter half is the reinterpret eval-slot fix on the optimizer branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: default<T[N]> assigned into a fixed array zeroes the whole array The make-struct arm zeroed max(elements, 1) strides; for default<T[N]> there are no elements and a stride is one element, so only the first element was cleared and `a = default<int[4]>` left the rest. The arm now zeroes the declared shape's size when no element is provided and reads the element type through the fixed-array chain for the empty-tuple case, as the interpreter does. Found by the regex rewrite: a set reset through default<> under -jit kept its old bits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: a pointer difference is inline arithmetic, not a slot call `p - q` called i_das_ptr_diff through the builtin's slot with the stride as a run-time argument, so the division never folded; the intrinsic row emits ptrtoint, sub and a division by the constant stride. The issue's loop goes from 0.32 s to 0.0045 s. Fixes GaijinEntertainment/daScript#4469 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: a global lookup is speculatable and hoists out of a loop jit_get_global_mnh depends on the name hash and the context alone and cannot trap, so it carries `speculatable`; LLVM then hoists a lookup under a branch inside a loop to the preheader. The issue's loop goes from 0.405 s to 0.042 s, beside 0.035 s for the solid_context build. Fixes GaijinEntertainment/daScript#4467 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: --jit-remarks flags print LLVM's optimization remarks --jit-remarks=, --jit-remarks-missed= and --jit-remarks-analysis= take a pass regex, set LLVM's -pass-remarks options once per process and force codegen past the DLL and object caches; under --jit-debug each remark carries the .das file and line. Fixes GaijinEntertainment/daScript#4470 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * jit: the typed-pointer aliasing check lands skipped The emitter attaches no alias information to typed pointers, by the language's own rule that a pointer is an address; the test documents the shape and stays skipped behind a flag. Refs GaijinEntertainment/daScript#4446 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * review: two checklists name their triggers and share one rule format The tests checklist's AOT-run rule described what test_aot compiles in a way that read as an example, and its opt-out rule named two of the four ways a file leaves the AOT set; a test that starts a daslang child now has its cache flags spelled out instead of sending the reader into the host's source. The dasLLVM checklist's routing rules take the list form the rest of the file uses, its fused tune rule splits into its two triggers, the child-artifact rule accepts a working directory inside the test's own directory, and the twinned-emitter definition is the closed list with a scalar criterion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * review: the version-bump rule covers a pass that rejects source A pass that newly rejects a shape leaves a cached module untouched, so the src/ast checklist's getVersion rule names that trigger beside the output change it already named. The AST-headers checklist cites its architecture section by anchor, drops the version-comment rule the root CLAUDE.md already carries, and narrows the binding-name rule to the names ast_handle.h itself computes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: the tree-sitter parser test runs on ubuntu-24.04 ubuntu-latest moved to 26.04, where the swift toolchain the parser test action installs fails to load libxml2.so.2 and swift-build exits 127. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> daScript-Commit: 21316206e9773ac0f07fa1fb060c4edab199c47c

Copilot AI balanced review requested due to automatic review settings November 7, 2026 06:05

Copilot started reviewing on behalf of borisbat November 7, 2026 06:05

View session

borisbat force-pushed the bbatkin/lookibed-jit branch from f667b0b to 63938f2 Compare November 7, 2026 06:12